Shape2Vec: semantic-based descriptors for 3D shapes, sketches and images

1 Convolutional neural networks have been successfully used to com2 pute shape descriptors, or jointly embed shapes and sketches in a 3 common vector space. We propose a novel approach that lever4 ages both labeled 3D shapes and semantic information contained 5 in the labels, to generate semantically-meaningful shape descrip6 tors. A neural network is trained to generate shape descriptors that 7 lie close to a vector representation of the shape class, given a vec8 tor space of words. This method is easily extendable to range scans, 9 hand-drawn sketches and images. This makes cross-modal retrieval 10 possible, without a need to design different methods depending on 11 the query type. We show that sketch-based shape retrieval using 12 semantic-based descriptors outperforms the state-of-the-art by large 13 margins, and mesh-based retrieval generates results of higher rele14 vance to the query, than current deep shape descriptors. 15